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The  Department  of  Energy  (doe)  contracts  with  private  companies  and 
educational  institutions  to  manage  and  operate  18  of  its  22  laboratories. 
These  are  cost  reimbursement  contracts  under  which  doe  pays  all  of  its 
contractors’  allowable  costs,  doe  can  also  provide  a  fee,  or  profit,  to  a 
contractor  for  managing  a  laboratory.  Responding  to  criticism  that  its 
historical  contracting  practices  were  costly  and  inefficient,  doe  switched 
to  performance-based  contracts  in  1994  as  part  of  its  contract  reform 
program.  Use  of  these  contracts  allows  doe  to  structure  each  contract  to 
provide  a  clear  statement  of  what  needs  to  be  accomplished — rather  than 
providing  broad  statements  of  work — and  to  rely  on  performance 
measures  to  evaluate  a  contractor’s  progress  toward  meeting  its 
objectives.  An  important  feature  of  performance-based  contracting  is 
providing  incentives,  including  fees,  to  the  contractor’s  achievement  of 
objectives  as  a  means  of  encouraging  superior  performance  and  lowering 
costs.  Concerned  about  the  progress  made  to  implement 
performance-based  contracting  at  the  national  laboratories,  you  asked  us 
to 

•  assess  the  status  of  performance-based  contracting  in  doe’s  national 
laboratory  contracts,  and 

•  identify  efforts  being  made  to  determine  the  impact  of  performance-based 
contracting. 


Results  in  Brief 


doe’s  use  of  performance-based  contracting  for  its  laboratories  is  in  a  state 
of  transition.  While  all  laboratory  contracts  we  examined  had  some 
performance-based  features,  we  found  wide  variance  in  the  number  of 
performance  measures  and  the  types  of  fees  negotiated.  About  half  of  the 
18  laboratory  contracts  have  performance  fees  to  encourage  superior 
performance — a  major  goal  of  performance-based  contracting.  Most  of  the 
remaining  laboratory  contracts  are  still  based  on  doe’s  traditional  fixed-fee 
arrangement  in  which  the  fees  are  paid  regardless  of  performance. 
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doe  has  not  evaluated  the  impact  of  performance-based  contracting  on  its 
laboratory  contractors  and,  as  a  result,  does  not  know  if  this  new  form  of 
contracting  is  achieving  the  intended  results  of  improved  performance  and 
lower  costs.  Specifically,  doe  has  not  determined  whether  giving  higher 
fees  to  encourage  superior  performance  by  its  laboratory  contractors  is 
advantageous  to  the  government,  although  we  recommended  in  1994  that 
doe  develop  criteria  for  measuring  the  costs  and  benefits  to  the 
government  of  using  higher  fees.1  Fees  for  the  laboratories  totaled  over 
$100  million  for  fiscal  year  1998.  While  the  contractors  were  unable  to  cite 
measurable  benefits  achieved  by  switching  to  performance-based 
contracting,  they  support  its  goals.  The  main  benefits  from 
performance-based  contracting  cited  by  laboratory  contractors  was  that  it 
has  helped  doe  clarify  what  it  expects  from  the  contractors  and  that  it  has 
improved  communication. 


Background 


doe  manages  the  largest  laboratory  system  of  its  kind  in  the  world.  Since 
the  early  days  of  the  World  War  II  Manhattan  Project,  doe’s  laboratories 
have  played  a  major  role  in  maintaining  U.S.  leadership  in  research  and 
development,  doe  is  responsible  for  ensuring  that  the  laboratory 
system — with  22  laboratories  in  14  states,  a  combined  budget  of  over 
$10  billion  a  year,  and  a  staff  of  about  60,000 — is  managed  in  an  effective, 
efficient,  and  economical  manner,  doe  contracts  with  educational 
institutions  and  private  sector  organizations  for  the  management  and 
operation  of  18  of  its  laboratories.  (App.  I  lists  doe’s  national  laboratories.) 
The  remaining  four  laboratories  are  staffed  by  federal  employees. 

doe  pays  its  laboratory  contractors  all  allowable  costs,  doe  can  also  pay 
contractors  a  separate  fee,  or  profit,  as  compensation  for  operating  the 
laboratories.  Fees  are  based  on  the  contract  value  and  the  technical 
complexity  of  the  work  to  be  performed  at  a  laboratory,  but  also  on  the 
degree  of  financial  liability  or  risk  that  a  contractor  is  willing  to  assume. 
Under  performance-based  contracting  principles,  fees  can  include  both  a 
fixed  amount  and  an  amount  that  is  linked  to  achieving  performance 
objectives.  One  of  doe’s  major  goals  in  performance-based  contracting  is 
to  develop  performance  objectives  for  each  contractor  that  are  specific, 
results-oriented,  measurable,  and  reflect  the  most  critical  activities. 


Energy  Management:  Modest  Reforms  Made  in  University  of  California  Contracts,  but  Fees  Are 
Substantially  Higher  (GAO/RCED-94-202,  Aug.  25, 1994). 
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Performance-Based 
Contracting  at  the 
National  Laboratories 
Is  an  Evolving  Process 


doe’s  implementation  of  performance-based  contracting  for  its 
laboratories  is  in  a  state  of  transition.  While  most  of  its  laboratory 
contracts  contain  some  performance-based  features,  the  contracts 
negotiated  by  doe  vary  from  contract  to  contract.  For  example,  doe  is 
incorporating  performance-based  features  in  all  of  its  laboratory  contracts, 
although  measures  vary  substantially  in  number,  ranging  from  a  low  of  7  in 
one  laboratory  contract  to  about  250  in  another.  Also,  doe  has  negotiated 
performance  fees  in  only  9  of  its  18  laboratory  contracts  because  the 
remaining  laboratories  are  still  operating  under  doe’s  traditional  approach 
in  which  fees  are  not  linked  to  performance.  We  found  that  similar 
laboratories  managed  by  similar  contractors  have  different  contracts.  The 
wide  diversity  of  contract  features  reflects  doe’s  philosophy  of  relying  on 
doe  field  units  to  tailor  contracts  to  local  conditions  and  contractors’ 
preferences. 


Developing  the  Right 
Performance  Measures  Is  a 
Challenge 


Since  introducing  performance-based  contracting  in  1994,  doe  and  its 
laboratory  contractors  have  struggled  to  find  the  right  mix  of  measures 
that  accurately  and  reliably  capture  the  contractors’  performance. 
According  to  doe  field  staff,  in  the  early  years  of  contract  reform,  doe 
encouraged  its  field  units  to  construct  as  many  measures  as  they  could, 
but  provided  limited  guidance  on  how  to  accomplish  this  task.  As  a  result, 
early  attempts  led  to  large  numbers  of  performance  measures.  A  large 
number  of  measures  diminishes  the  importance  of  any  single  measure, 
whereas  a  small  number  results  in  measures  that  are  too  broad  to  be 
meaningful.  For  example,  a  doe  field  official  told  us, 


“The  original  guidance  from  doe  Headquarters  was  to  [develop 
performance  measures]  as  much  as  possible.  Unfortunately,  there  was 
inadequate  guidance  on  how  to  do  this. . . .  The  number  of  performance 
measures  ...  is  too  large.  However,  if  we  fail  to  cover  an  activity  [with  a 
measure]  the  contractor  may  not  give  the  attention  needed  to  the 
activity.” 


doe  and  its  laboratories  are  still  attempting  to  develop  the  right  number  of 
measures.  For  example,  we  found  that  the  number  of  performance 
measures  in  the  laboratory  contracts  we  examined  ranged  from  a  low  of  7 
measures  at  the  Ames  Laboratory  in  Iowa  to  about  250  at  the  Idaho 
National  Engineering  and  Environmental  Laboratory  in  Idaho. 

doe  and  its  contractors  are  also  working  to  develop  measures  that  reliably 
address  the  most  important  activities  of  the  laboratories.  According  to  a 
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Types  of  Fees  Paid  to 
Contractors  Vary  Widely 


field  official,  doe’s  early  attempts  at  developing  performance  measures 
resulted  in  contractors  focusing  only  on  those  activities  that  were  tied  to 
performance  fees,  while  neglecting  other  important  activities.  Another  doe 
site  official  stated, 

“[P]erformance-based  contracting  tends  to  focus  too  much  on  the 
monetary  reward . . .  and  less  on  an  analysis  of  performance.  The  incentive 
at  the  labs  should  be  [for]  good  science,  not  more  dollars.” 

Developing  the  right  number  and  type  of  performance  measures  is  an 
evolving  process  between  doe  and  its  contractors.  Most  doe  and 
contractor  representatives  told  us  that  they  are  making  progress  in  finding 
measures  that  accurately  and  reliably  reflect  performance,  particularly  in 
management  and  operations  activities.  Measuring  a  contractor’s 
performance  in  science  and  technology  is  more  difficult.  Science  and 
technology  measures  are  broader  in  scope  and  typically  rely  on  peer 
reviews  and  a  contractor’s  self-assessment  for  evaluating  performance. 


Although  performance  fees  are  a  major  feature  of  performance-based 
contracting,  only  9  of  the  Department’s  18  laboratory  contracts  have  them. 
Nine  of  the  remaining  laboratory  contracts  operate  under  doe’s  traditional 
fixed-fee  arrangement,  and  one  laboratory  contract  has  no  fee.  Fixed  fees 
are  earned  regardless  of  performance  and  were  commonly  used  before 
doe  adopted  performance-based  contracting  as  its  normal  business 
practice.  Appendixes  I  and  II  summarize  laboratory  fee  arrangements  and 
illustrate  the  wide  variety  of  fee  arrangements  in  use.  In  commenting  on  a 
draft  of  this  report,  doe  said  that  by  the  end  of  calendar  year  1999,  the 
majority  of  laboratory  contracts  that  provide  fees  will  have 
performance-based  fee  structures. 

Performance  fees  were  introduced  as  a  way  of  encouraging  superior 
performance  and  can  include  an  incentive  and  an  award  fee.  An  incentive 
fee  is  usually  applied  to  activities  for  which  progress  can  be  accurately 
measured,  for  example,  cleaning  up  40  barrels  of  toxic  waste  within  a 
prescribed  period  of  time.  An  award  fee  is  usually  applied  to  tasks  that  are 
harder  to  measure  and  require  a  more  subjective  judgment  of 
performance,  for  example,  assessing  a  contractor’s  attention  to 
community  relations.  Performance  fees  represent  the  amount  of  a 
contractor’s  total  fee  placed  “at  risk”  since  the  fee  that  could  be  earned  is 
determined  by  how  well  the  contractor  performs. 
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As  the  following  examples  show,  some  laboratory  contracts  include  both 
types  of  performance  fees,  while  others  rely  solely  on  an  incentive  fee  or 
an  award  fee.  Still  others  have  neither  and  use  only  fixed  fees. 

•  At  the  Sandia  National  Laboratories  in  New  Mexico  and  the  Oak  Ridge 
National  Laboratory  in  Tennessee,  doe  negotiated  fixed-fee  contracts. 

Both  of  these  laboratories  are  operated  by  subsidiaries  of  the  Lockheed 
Martin  Corporation — a  for-profit  company,  doe  officials  told  us  they  were 
confident  that  incentive  fees  were  not  needed  for  these  laboratories 
because  the  existing  Lockheed  Martin  contractors’  performance  is 
superior  and  introducing  incentive  fees  might  distract  the  contractors 
from  performing  all  essential  work. 

•  At  the  Idaho  National  Engineering  and  Environmental  Laboratory  in  Idaho, 
operated  by  Lockheed  Martin  Idaho  Technologies  Company,  doe  uses  a 
combination  of  fixed,  incentive,  and  award  fees,  doe  officials  told  us  that 
incentive  fees  were  used  because  of  the  many  different  tasks  that  could  be 
identified  and  measured,  but  that  award  fees  were  also  needed  to  assess 
activities  that  required  more  subjective  judgments. 

•  At  the  Stanford  Linear  Accelerator  Center  in  California,  operated  by 
Stanford  University,  doe  negotiated  a  no-fee  contract,  the  only  such 
arrangement  in  the  laboratory  system.  According  to  doe,  the  laboratory 
contractor  does  not  want  a  fee  for  operating  this  laboratory  because  a  fee 
would  not  motivate  performance  and  may  be  a  detriment  to  the  conduct  of 
outstanding  science,  which  is  the  primary  mission  of  this  laboratory. 

•  The  Lawrence  Berkeley  National  Laboratory  and  Lawrence  Livermore 
National  Laboratory  in  California  and  the  Los  Alamos  National  Laboratory 
in  New  Mexico  are  operated  by  the  University  of  California.  The  contracts 
contain  a  fixed  fee  and  an  incentive  fee  for  meeting  expectations,  plus 
another  amount  for  exceeding  expectations. 

A  senior  doe  official  acknowledged  the  variability  in  laboratory  contracts 
but  said  that  imposing  uniform  practices  throughout  the  laboratory  system 
would  not  necessarily  improve  the  overall  performance  and  accountability 
of  the  contractors.  According  to  doe  and  laboratory  officials,  there  are 
several  reasons  for  the  variability  in  the  contracts.  First,  the  laboratories 
engage  in  different  activities  with  different  levels  of  technical 
complexities.  Second,  some  contractors  are  willing  to  assume  greater 
financial  risk  or  liability  and  thus  expect  a  higher  or  different  fee 
arrangement.  Finally,  doe  field  officials  who  negotiate  the  contracts 
employ  features  that  they  believe  are  best  suited  for  their  particular 
circumstances.  However,  we  found  that  similar  laboratories  operated  by 
similar  contractors  have  different  fee  arrangements.  For  example,  both  the 
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Lawrence  Berkeley  and  Argonne  national  laboratories  have  similar 
research  missions  and  are  both  managed  by  university  contractors. 
However,  Lawrence  Berkeley’s  contractor,  the  University  of  California, 
works  under  a  fixed-fee  plus  performance  fee  arrangement,  while 
Argonne ’s  contractor,  the  University  of  Chicago,  works  under  a 
performance  fee  arrangement  only. 

We  also  found  substantial  variations  in  contracting  philosophy  among  doe 
field  officials,  doe  relies  on  field  units  to  negotiate  its  contracts,  including 
whether  to  use  performance-based  fees,  and  how  performance  objectives 
and  measures  will  be  accomplished.  Some  of  these  officials  told  us  that 
performance  fees  are  important  motivators,  while  others  said  performance 
fees  can  distract  the  contractor  from  other  important  work. 

In  commenting  on  a  draft  of  this  report,  doe  provided  us  with  additional 
reasons  for  the  variability  in  contracts,  including  the  timing  of  when 
contractors  first  converted  to  performance-based  contracting,  the  nature 
of  the  proposals  received  in  competitive  awards,  and  the  negotiated  terms 
in  contract  extensions.  In  addition,  doe  cited  other  motivations  for 
laboratory  contractors,  such  as  their  reputations  in  the  scientific 
community  and  contract  extensions. 


The  Impact  of 
Performance-Based 
Contracting  Remains 
Unknown 


doe’s  guidance  states  that  the  purpose  of  performance-based  contracting  is 
to  obtain  better  performance  or  lower  costs  or  both,  doe  has  not  analyzed 
the  impact  of  performance-based  contracting  on  its  laboratory 
contractors.  As  a  result,  it  has  not  determined  whether  performance-based 
contracting  is  achieving  the  intended  objectives  of  reducing  costs  and 
improving  performance. 


doe  officials  told  us  that  the  amounts  of  fees  paid  to  laboratory 
contractors  have  generally  increased  with  the  implementation  of 
performance-based  contracting  but  that  it  is  difficult  to  determine  the 
return  on  this  investment  since  contractors  are  also  assuming  more  risk  or 
liability  for  costs  previously  paid  by  doe.  Increased  liabilities  include  costs 
due  to  a  failure  to  exercise  prudent  business  judgment  on  the  part  of  the 
contractor’s  managerial  personnel,  doe  has  not  analyzed  the  relative  costs 
and  benefits  to  the  government  of  using  higher  fees  in  performance-based 
contracts.  We  previously  recommended  that  doe  ensure  that  the  fees  paid 
to  contractors  for  incurring  increased  financial  risks  are  cost-effective  by 
developing  criteria  for  measuring  the  costs  and  benefits  to  the  government 
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of  this  approach.2  doe  officials  told  us  that  while  they  have  not  conducted 
a  comprehensive  cost-benefit  analysis  of  fees,  they  try  to  negotiate  fees 
that  make  sense  for  individual  contracts,  taking  into  account  the  financial 
risks  and  incentives  needed  to  motivate  performance.  Without  such  an 
overall  analysis,  however,  it  is  difficult  to  determine  the  value  to  the 
government  of  the  over  $100  million  spent  on  contractor  fees  for  fiscal 
year  1998. 

Although  doe  has  not  assessed  the  impact  of  performance-based 
contracting,  limited  reviews  have  found  both  progress  and  problems,  as 
these  examples  show: 

•  Since  1997,  doe’s  Office  of  Inspector  General  has  issued  three  reports  on 
problems  the  Department  had  in  implementing  performance-based 
incentives  at  three  facilities  (one  of  which  was  a  laboratory).3  Problems 
reported  by  the  Inspector  General  included  contracts  with  poorly 
developed  performance  measures  and  fees  that  were  paid  to  contractors 
before  agreement  was  reached  on  the  performance  incentives. 

•  In  1997,  doe’s  Office  of  Procurement  issued  a  report  on  the  use  of 
performance-based  incentives.  The  report  noted  that  the  use  of  incentives 
has  been  effective  in  directing  contractors’  attention  to  performance 
outcomes  and  has  improved  communications  concerning  performance 
expectations.  The  report  also  noted  that  doe  field  units  are  improving  the 
quality  of  their  contracts.  However,  the  report  pointed  out  that 
implementation  was  sometimes  inconsistent  and  that  performance 
objectives  sometimes  were  overly  focused  on  process  milestones  rather 
than  on  outcomes,  doe’s  laboratories  were  not  the  focus  of  this  review, 
however.4 

•  Our  July  1998  report  on  doe’s  performance-based  incentive  contracts 
noted  that  the  Department  had  taken  steps  to  correct  many  of  the 
problems  cited  in  the  Inspector  General’s  reports,  including  issuing 
guidance,  conducting  training,  and  incorporating  lessons  learned  into 
fiscal  year  1998  contract  incentives.6  We  noted  that  although  doe 


2GAO/RCED-94-202,  Aug.  25, 1994. 

3Inspection  of  the  Performance  Based  Incentive  Program  at  the  Richland  Operations  Office 
(doe/IG-0401,  Mar.  10,  1997);  Audit  of  the  Contractor  Incentive  Programs  at  the  Rocky  Flats 
Environmental  Technology  Site  (doe/IG-0411,  Aug.  13,  1997);  and  Inspection  Report:  The  Fiscal  Year 
1996  Performance  Based  Incentive  Program  at  the  Savannah  River  Operations  Office  (doe-INS-O-98-03, 
May  1998). 

Contract  Reform  Self  Assessment  Report,  Office  of  Contract  Reform  and  Privatization,  DOE  (Sept 
1997). 

department  of  Energy:  Lessons  Learned  Incorporated  Into  Performance-Based  Incentive  Contracts 
(GAO/RCED-98-223,  July  15, 1998). 
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maintained  that  its  performance-based  incentives  have  been  effective  in 
achieving  the  desired  end  results,  it  had  not  been  clear  whether  these 
successes  were  due  to  performance-based  incentives  or  to  an  increased 
emphasis  on  program  management. 

None  of  these  assessments  focused  exclusively  on  laboratory  contracts.  In 
our  discussions,  doe  field  staff  generally  credited  performance-based 
contracting  with  improving  their  ability  to  set  expectations  for  the 
Department’s  laboratories,  and  several  laboratory  contractors  concurred 
that  this  was  a  benefit.  In  addition,  both  doe  and  laboratory  officials  cited 
improved  communication  as  a  benefit  of  performance-based  contracting. 
Laboratory  contractors  also  credited  doe  for  focusing  its  oversight  on 
evaluating  results  and  away  from  dwelling  on  strict  compliance  with  doe’s 
rules  and  regulations.  In  addition,  contractors  told  us  they  have  increased 
productivity  and  lowered  costs,  especially  for  the  support  and  overhead 
functions.  However,  most  of  these  officials  also  said  that  these  advances 
were  more  the  result  of  other  initiatives,  such  as  internal  streamlining 
actions,  than  of  performance-based  contracting. 


Conclusions 


doe  and  its  laboratory  contractors  told  us  that  they  are  committed  to 
making  performance-based  contracting  work  effectively  and  that  the 
contracts  are  including  more  specific  and  reliable  performance  measures. 
However,  since  doe  has  not  evaluated  the  impact  of  performance-based 
contracting  on  its  laboratories — owing  in  part  to  the  wide  variance  in  fee 
arrangements— there  is  limited  evidence  on  how  performance  fees  ensure 
a  high  level  of  performance  by  contractors  at  lower  cost.  As  a  result,  doe 
cannot  show  how  the  higher  fees  it  is  paying  to  contractors  under 
performance-based  contracting  are  of  value  to  the  government  and  to  the 
taxpayers. 

We  previously  recommended  that  the  Secretary  of  Energy  ensure  that  the 
fees  paid  to  contractors  for  incurring  increased  financial  risk  are 
cost-effective  by  developing  criteria  for  measuring  the  costs  and  benefits 
to  the  government  of  this  approach,  doe  did  not  implement  our 
recommendation  and  has  no  plans  to  measure  the  overall  costs  and 
benefits  of  performance-based  contracting  for  its  laboratories,  doe 
officials  maintain  that  performance-based  contracting  is  working,  but  this 
is  based  on  anecdotal  evidence.  Moreover,  the  fees  doe  negotiates  are 
based  on  its  best  judgment  of  what  is  needed  to  motivate  contractors  and 
to  compensate  them  for  increased  risk,  but  doe’s  evidence  is  based 
primarily  on  non-laboratory  contractors,  and  doe  has  not  quantified  the 
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value  of  the  increased  risk  assumed  by  contractors  under 
performance-based  conditions. 


Recommendation 


Because  doe  does  not  know  whether  performance-based  contracting  is 
improving  performance  at  lower  cost  at  its  national  laboratories  and 
because  our  previous  recommendation  to  develop  criteria  for  measuring 
the  costs  and  benefits  of  paying  fees  to  contractors  for  incurring  increased 
financial  risk  was  not  implemented,  we  recommend  that  the  Secretary  of 
Energy  evaluate  the  costs  and  benefits  from  using  performance-based 
contracting  at  the  national  laboratories.  While  we  recognize  that  each 
laboratory  contract  is  individually  negotiated,  doe  should  nevertheless 
ensure  that  the  fees  it  provides  to  motivate  contractors  and  to  compensate 
them  for  increased  financial  risk  is  based  on  an  analysis  of  costs  and 
benefits.  The  need  for  this  type  of  evaluation  is  consistent  with  the 
principles  of  the  Government  Performance  and  Results  Act  of  1993  that 
require  agencies  to  measure  outcomes  against  their  goals.6 


Agency  Comments 


We  provided  a  draft  of  this  report  to  doe  for  review  and  comment,  doe 
disagreed  with  our  conclusion  on  the  need  for  determining  the  costs  and 
benefits  of  the  fees  it  has  negotiated  with  its  laboratory  contractors,  doe 
noted  that  its  performance-based  contracting  experience  is  in  transition 
but  that  its  evaluations  show  that  performance-based  contracting  is 
working.  We  acknowledge  in  our  report  that  doe’s  evaluations  of 
performance-based  contracting  show  promise,  but  we  also  point  out  that 
these  evaluations  did  not  focus  on  the  laboratories’  experiences  with 
performance-based  contracting.  Because  of  this  limitation  and  because  of 
the  higher  fees  being  negotiated  with  the  laboratories,  we  continue  to 
believe  it  is  desirable  for  doe  to  determine  if  its  performance-based 
contracting  is  improving  performance  at  lower  cost. 

doe  also  commented  that  the  variability  we  found  in  performance-based 
laboratory  contracts  reflects  many  different  factors,  including  differences 
in  the  scope  of  work,  the  type  of  contractor,  and  the  experiences  the 
laboratories  have  with  performance-based  contracting  features.  Our  report 
described  the  reasons  for  the  variability  in  laboratory  contracts,  and  we 
have  included  the  additional  reasons  provided  in  doe’s  comments.  We  also 
agree  that  doe’s  use  of  performance-based  contracting  is  evolving  and  that 
the  variability  we  found  in  laboratory  contracts  (principally  in 


“The  Results  Act  applies  to  agencies  as  defined  in  5  U.S.C.  306(f),  which  generally  covers  executive 
departments,  government  corporations,  and  independent  establishments. 
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performance  measures  and  fee  arrangements)  is  in  part  due  to  an  ongoing 
learning  process  associated  with  the  transition  to  performance-based 
contracting. 

doe  also  raised  a  number  of  issues  regarding  the  use  of  fees  in  its 
laboratory  contracts  and  strongly  defended  its  use  of  performance  fees. 

We  agree  with  many  of  doe’s  observations  on  the  use  of  performance  fees, 
and  we  are  not  suggesting  that  doe  should  abandon  its  performance-based 
approach  or  that  it  should  eliminate  performance-based  fees  in  its 
laboratory  contracts.  It  is  also  not  our  intent  to  show  that 
performance-based  contracting  should  be  abandoned  if  its  impacts  on  the 
laboratories  cannot  be  measured.  We  do  believe,  however,  that  effective 
implementation  of  performance-based  contracting  provisions  is  dependent 
on  the  ability  to  support  the  fee  amounts  paid  through  a  cost  and  benefit 
analysis. 

doe  also  provided  a  number  of  clarifications  that  we  have  incorporated  in 
our  report  as  appropriate.  Appendix  HI  includes  the  full  text  of  doe’s 
comments  and  our  response. 


Our  review  was  performed  from  September  1998  through  April  1999  in 
accordance  with  generally  accepted  government  auditing  standards.  See 
appendix  IV  for  a  description  of  our  scope  and  methodology. 

As  arranged  with  your  offices,  unless  you  publicly  announce  its  contents 
earlier,  we  plan  no  further  distribution  of  this  report  until  30  days  after  the 
date  of  this  letter.  At  that  time,  we  will  send  copies  to  Bill  Richardson, 
Secretary  of  Energy,  and  Jacob  J.  Lew,  Director,  Office  of  Management 
and  Budget.  We  will  make  copies  available  to  others  on  request. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  call  me  at 
(202)  512-3841.  Major  contributors  to  this  report  were  Gary  R.  Boss  and 
Tom  Kingham. 


Susan  D.  Kladiva 
Associate  Director,  Energy, 
Resources,  and  Science  Issues 


Page  10 


GAO/RCED-99  141  Performance-Based  Contracting 


Page  11 


GAO/RCED  99-141  Performance  Based  Contracting 


Contents 


Letter 

i 

Appendix  I 

Contract  Amount  and 

Fees  Earned  by  DGE’s 

Laboratory 

Cont  ractors  in  Fiscal 

Year  1998 

14 

Appendix  II 

Status  of  DOE 

Laboratory  Contracts 
as  of  March  30, 1999 

16 

Appendix  III 

Comments  From  the 

Department  of  Energy 

18 

Appendix  IV 

Scope  and 

Methodology 

29 

Abbreviations 

doe  Department  of  Energy 

gao  General  Accounting  Office 

r&d  research  and  development 


Page  12 


GAO/RCED-99  141  Performance-Based  Contracting 


Page  13 


GAO/RCED-99-141  Performance-Based  Contracting 


Appendix  I _ _ _ 

Contract  Amount  and  Fees  Earned  by  DOE’s 
Laboratory  Contractors  in  Fiscal  Year  1998 


Laboratory/contractor 

Contract  amount 
(millions)3 

Fixed  or  base  feeb 

Performance  fee0 

Total  fee  earned 

Argonne  National  Laboratory/University 
of  Chicago 

$466.9 

$3,425,000 

$3,425,000 

Brookhaven  National 
Laboratory/Brookhaven  Science 

Associates 

385.9 

3,574,000 

3,574,000 

Idaho  National  Engineering  and 
Environmental  Laboratory/Lockheed 

Martin  Idaho  Technologies  Company 

578.7 

9,848,000 

9,848,000 

Lawrence  Berkeley  National 
Laboratory/University  of  California 

320.0 

420,000 

1,063,780 

1,483,780 

Lawrence  Livermore  National 
Laboratory/University  of  California 

1,100.0 

1,680,000 

4,482,000 

6,162,000 

Los  Alamos  National 

Laboratory/University  of  California 

1,345.0 

2,100,000 

5,550,000 

7,650,000 

Oak  Ridge  National 

Laboratory/Lockheed  Martin  Energy 
Research 

475.6 

7,220,000 

7,220,000 

Pacific  Northwest  Laboratory/Battelle 
Memorial  Institute 

461.0 

5,600,000 

5,600,000 

Sandia  National  Laboratories/Sandia 

Corp.  (Lockheed  Martin) 

1,397.6 

14,347,000 

14,347,000 

Ames  Laboratory/Iowa  State  University 

25.4 

TBDd 

TBDd 

Fermi  National  Accelerator 
Laboratory/University  Research 

Associates,  Inc. 

279.6 

2,750,000 

2,750,000 

National  Renewable  Energy 
Laboratory/Midwest  Research  Institute 

199.4 

3,522,500 

3,522,500 

7,045,000 

Princeton  Plasma  Physics 
Laboratory/Princeton  University 

59.4 

10,000 

10,000 

Stanford  Linear  Accelerator 
Center/Stanford  University 

187,0 

Thomas  Jefferson  National  Accelerator 
Facility/Southeastern  University 

Research  Associates,  Inc. 

70.2 

1,874,633 

1 ,874,633e 

Bettis  Atomic  Power  Laboratory/Bechtel 
Group 

305.1 

8,686,000 

8,686,000 

Knolls  Atomic  Power  Laboratory/KAPL, 

Inc.  (Lockheed  Martin) 

265.0 

7,300,000 

7,300,000 

Savannah  River  Technology 
Centerf/Westinghouse  Savannah  River 

Co. 

1,248.0 

51,570,100 

51,570,100 

(Table  notes  on  next  page) 
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Contract  Amount  and  Fees  Earned  by  DOE’s 
Laboratory  Contractors  in  Fiscal  Year  1998 


aContract  amounts  for  fiscal  year  1998  include  some  estimates. 

bA  base  fee,  used  in  a  performance  fee  contract,  is  the  part  of  the  fee  not  at  risk  and  is  similar  to  a 
fixed  fee, 

CA  performance  fee  can  be  either  an  award  fee  or  an  incentive  fee  or  a  combination. 

dThe  fee  is  to  be  determined  after  DOE  reviews  of  the  contractor’s  self-assessment  and  after  the 
issuance  of  final  DOE  reports. 

eThe  fee  is  considered  a  management  allowance,  which  is  similar  to  a  fixed  fee. 

fThe  contract  and  fee  amounts  shown  are  for  the  entire  Savannah  River  Site,  including  the 
Savannah  River  Technology  Center. 

Source:  GAO  based  on  data  from  DOE's  headquarters  and  operations  offices. 
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Status  of  DOE  Laboratory  Contracts  as  of 
March  30,  1999 


Laboratory/contractor 

Type  and  status  of  contract 

Argonne  National  Laboratory/University  of  Chicago 

Incentive-fee  contract.  DOE  plans  to  extend  this  contract  5  years,  but  is 
renegotiating  to  make  it  consistent  with  the  federal  acquisition 
regulations  format  and  to  incorporate  all  contract  reform  features, 
including  performance-based  provisions. 

Brookhaven  National  Laboratory/Brookhaven  Science 
Associates 

Fixed-fee  contract.  The  new  contract  was  signed  in  January  1 998  with  a 
fixed  fee  through  September  1998.  DOE  is  still  negotiating  the  contract 
for  fiscal  year  1999.  DOE  plans  to  negotiate  a  performance  fee. 

Idaho  National  Engineering  and  Environmental 
Laboratory/Lockheed  Martin  Idaho  Technologies  Company 

Incentive-fee  contract.  DOE  plans  to  recompete  this  contract  in  fiscal 
year  1999.  The  current  contractor,  Lockheed  Martin,  announced  it  will 
not  bid. 

Lawrence  Berkeley  National  Laboratory/University  of 
California 

Incentive-fee  contract.  The  amount  of  the  annual  available  fee  remains 
the  same  for  each  year  of  the  5-year  contract. 

Lawrence  Livermore  National  Laboratory/University  of 
California 

Incentive-fee  contract.  The  amount  of  the  annual  available  fee  remains 
the  same  for  each  year  of  the  5-year  contract. 

Los  Alamos  National  Laboratory/University  of  California 

Incentive-fee  contract.  The  amount  of  the  annual  available  fee  remains 
the  same  for  each  year  of  the  5-year  contract. 

Oak  Ridge  National  Laboratory/Lockheed  Martin  Energy 
Research 

Fixed-fee  contract.  DOE  is  recompeting  this  contract.  DOE  plans  to 
convert  this  contract  to  a  performance  fee.  Lockheed-Martin  announced 
that  it  will  not  bid  as  a  prime  contractor  on  the  new  contract.  The  other 
two  contracts  for  this  site — for  environmental  cleanup  and 
production—- are  performance-fee 

Pacific  Northwest  Laboratory/Battelle  Memorial  Institute 

Incentive-fee  contract.  DOE  converted  this  contract  from  a  fixed-fee  to 
an  incentive-fee  type  and  made  available  $7.1  million  in  potential  fees 
geared  to  incentives  in  four  areas— science  and  technology  excellence, 
operational  excellence,  leadership  and  management,  and  community 
relations. 

Sandia  National  Laboratories/Sandia  Corp.  (Lockheed  Martin)  Fixed-fee  contract.  The  contract  that  expired  in  September  1998  was 

renegotiated  and  extended  noncompetitively  for  5  years.  The  new 
contract  remains  a  fixed-fee  arrangement  but  now  includes  performance 
objectives,  measures,  and  criteria.  DOE  decided  that  the  contractor's 
superior  performance  could  be  sustained  with  a  fixed  fee. 

Ames  Laboratory/Iowa  State  University 

Incentive-fee  contract.  DOE  is  renegotiating  this  contract  and  plans  to 
extend  noncompetitively  for  5  years.  DOE  plans  to  make  the  contract 
consistent  with  the  federal  acquisition  regulations  format  and  to 
incorporate  all  contract  reform  conditions,  including  performance-based 
provisions. 

Fermi  National  Accelerator  Laboratory/University  Research 
Associates,  Inc. 

Fixed-fee  contract.  DOE  has  not  announced  whether  it  will  recompete  or 
extend  this  contract.  DOE  rates  the  contractor's  performance  as 
outstanding. 

National  Renewable  Energy/Midwest  Research  Institute 

Award-fee  contract.  DOE  recompeted  this  contract  in  1998.  The  new 
contract  was  effective  on  Oct.  1, 1998,  and  is  fixed  fee  until  March  1999, 
at  which  time  DOE  intends  to  includes  an  award  fee  for  the  remainder  of 
the  contract  period, 

Princeton  Plasma  Physics  Laboratory/Princeton  University 

Fixed-fee  contract.  The  contractor  did  not  want  any  fee,  but  DOE 
negotiated  a  small  fee  of  $10,000. 

(continued) 
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Status  of  DOE  Laboratory  Contracts  as  of 
March  30,  1999 


Laboratory/contractor 

Type  and  status  of  contract 

Stanford  Linear  Accelerator  Center/Stanford  University 

No-fee  contract.  The  contract  term  ended  on  March  31,  1998,  and  was 
extended  noncompetitively  on  a  month-by-month  basis  during 
negotiations  to  incorporate  performance-based  incentives.  The  contract 
was  then  extended  noncompetitively  for  5  years  in  January  1999.  The 
contract  includes  performance  measures  and  expectations,  but  no  fee. 

Thomas  Jefferson  National  Accelerator  Facility/Southeastern 
University  Research  Associates,  Inc 

No  fee  contract  (with  management  allowance).  The  contract  is  currently 
being  renegotiated  so  that  it  can  be  extended  noncompetitively  for  5 
years,  DOE  plans  the  new  contract  to  be  a  fixed-fee  arrangement. 
Objectives  of  the  negotiations  are  to  structure  the  contract  to  be 
consistent  with  the  federal  acquisition  regulations  format  and  to 
incorporate  all  contract  reform  conditions,  including  performance-based 
features. 

Bettis  Atomic  Power  Laboratory/Bechtel  Group 

Fixed-fee  contract.  The  contract  was  recompeted  in  1998.  The  new 
contractor  was  selected  (the  Bechtel  Group)  but  the  incumbent 
contractor,  Westinghouse  Electric  Corporation,  protested  the  award.  The 
existing  contract  extended  non-competitively  pending  the  result  of  a  bid 
protest  to  GAO.  The  bid  protest  was  denied  by  GAO.  The  new  contract 
with  Bechtel  was  effective  February  1 ,  1999. 

Knolls  Atomic  Power  Laboratory/KAPL,  Inc.  (Lockheed 

Martin) 

Fixed-fee  contract. 

Savannah  River  Technology  CenterAA/estinghouse  Savannah 
River  Co. 

Incentive-fee  contract. 

Source:  GAO  based  on  data  from  DOE’s  operations  offices. 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


Department  of  Energy 

Washington.  DC  20585 


April  22 ,  1999 


MEMORANDUM  FOR 


FROM: 


SUBJECT:  RESPONSE  TO  DRAFT  REPORT  ENTITLED,  '’NATIONAL 

LABORATORIES:  DOE  NEEDS  TO  ASSESS  THE  IMPACT  OF 
USING  PERFORMANCE-BASED  CONTRACTS" 


Thank  you  for  the  opportunity  to  review  your  draft  report  on  ’’issues  relating  to  performance- 
based  contracting  in  the  national  laboratories."  We  appreciate  the  difficulties  in  providing  an 
assessment  of  this  recent  initiative  as  well  as  the  complexity  associated  with  consideration  of 
such  disparate  contracts.  However,  we  believe  that  the  draft  could  be  significantly  improved  by 
1)  correcting  certain  misimpressions,  2)  providing  greater  context  to  certain  observations 
3)  clarifying  the  occasional  innuendo,  and  4)  providing  greater  recognition  of  the  Departments 
regulations  relating  to  fee. 

Our  comments  are  rather  brief  given  the  short  time  available  for  review.  We  would  be  glad  to 
expand  upon  them  at  your  convenience. 


SUSAN  D.  KLADIVA 
ASSOCIATE  DIRECTOR  FOR  ENERGY 
RESOURCES,  AND  SCIENCE  ISSUES 
GENERAL  ACCOUNTING  OFHi 


RICHARD  H.  HOPFC 
DIRECTOR  OF  PROCURE* 

AND  ASSISTANCE  MANAGEMENT 
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COMMENTS  ON  DRAFT  GAO  REPORT 

GAO/RCED-99-141 

See  comment  1. 

1.  Page  I.  If  1.  a)  The  report  states  that  in  response  to  certain  criticisms  DOE  switch  to 

performance-based  contracts  in  1994.  A  more  accurate  statement  w'ould 
be  that  in  1994  DOE  proposed  and  adopted  a  series  of  changes  to  its 
management  and  operating  contracts.  The  initiative  was  based  on  a  sense 
that  consistent  with  government  "reinvention"  themes,  these  contracts 
could  be  improved  and  was  supported  by  input  from  a  broad  range  of 
sources  within  and  without  the  Department.  One  of  the  many  changes 
pursued  was  the  concept  of  applying  performance-based  contracting 
techniques. 

See  comment  2. 

b)  The  report  states  that  M[T]hese  contracts  require  DOE  to  provide  a  clear 
picture  . .  .tt  The  contracts  don’t  require  this.  It  would  be  more  accurate 
to  state  that  the  concept  of  a  performance-based  management  contract 
includes  the  use  of  clearly  stated  performance  objectives  and  measures 

See  comment  1 . 

c)  The  report  states  that  "an  important  feature  of  performance-based 
contracts  is  linking  fees  . . It  would  be  more  accurate  to  state  that  ’’an 
important  element  of  performance-based  contracting  is  providing  incentives 
for  contractors  to  . . .  Linking  available  fees  to  the  achievement  of 
performance  objectives  is  an  obvious  and  potentially  effective  example  of 
such  an  incentive." 

Now  on  p.  1 . 

See  comment  1. 

2.  Paffe  2,  A 1.  a)  The  report  states  that  "about  half  of  the  1 9  laboratory  contracts  have 

performance  fees  to  encourage  superior  performance  -  a  major  goal  of 
performance-based  contracting."  A  more  accurate  statement  would  be  that 
by  the  end  of  the  calendar  year  FY  99  of  the  majority  of  laboratory 
contracts  that  provide  fee  will  have  performance-based  fee  structures.  The 
remaining  contracts  are  fixed-fee  arrangements. 

See  comment  3. 

3.  Page  2.  2.  a)  It  is  not  accurate  to  state  that  "DOE  has  not  evaluated  the  impact  of 

performance-based  contracting."  There  have,  in  fact,  been  a  number  of 
such  evaluations. 

See  comment  4. 

b)  It  is  inapt  to  state  that  "DOE"  has  not  determined  whether  giving  higher 
fee  to  encourage  superior  performance  is  advantageous . . The  provision 
of  fee  is  appropriate  for  a  number  of  purposes,  not  just  to  obtain  "superior 
performance."  For  example,  one  of  the  purposes  of  fee  under  a  cost- 
reimbursement  contract  is  to  recognize  financial  risk  associated  with  wrork 
performance.  Such  risks  were,  as  a  product  of  DOE's  contract  reform 
effort,  broadened  for  contractors  managing  DOE  labs.  Another  purpose  is 
to  recognize  the  accomplishment  of  contract  objectives,  regardless  of 
adjectival  description.  Further,  the  reference  to  the  1994  GAO 
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See  comment  5. 


See  comment  6. 


Now  on  p.  6. 
See  comment  7. 


recommendation  is  inapt  -  it  was  associated  not  with  performance  fees,  but 
rather  with  increased  financial  risk.  Ironically,  DOE  has  increased 
performance  and  financial  risk  under  these  contracts  based  on  various 
recommendations,  including  those  of  the  GAO,  that  they  be  brought  more 
in  line  with  other  Federal  contracts.  However,  it  would  appear  that  GAO 
now,  by  innuendo,  criticizes  the  consequences  of  such  changes.  Further, 
the  reference  to  increased  fees  is  in  large  part,  confused  with  other 
initiatives  having  little  to  do  with  performance-based  concepts.  Finally, 
given  the  admitted  inability  of  the  GAO  auditors  to  suggest  an  appropriate 
cost-benefit  model,  as  well  as  the  absence  of  such  model  as  applied  to  any 
other  Federal  fee  scheme  we  question  the  realism  of  the  recommendation. 

c)  Suggest  that  the  reference  to  laboratory  fees  in  fi  seal  year  1 998  be  put  in 

context:  "this  amount  represents  less  than _ %  of  the  total  value  of 

laboratory  contracts,  making  them  among  the  lowest  provided  by  any 
Federal  agency. 

Page  3,  S  3>  The  report  notes  that  "the  type  of  contract  varies  from  contractor  to 

contractor".  The  statement  is  inaccurate.  The  contract  "type"  for  DOE 
management  and  operating  contracts  is  fairly  standard.  Further,  to  the 
extent  that  there  are  variations  they  are  "from  contract  to  contract"  not 
"contractor  to  contractor."  The  variations  that  the  report  appears  to  refer 
to  relate  to  the  number  of  performance  objectives  measures  and  the  use  of 
various  fee  approaches.  There  appears  to  be  an  implied  criticism  of  the 
variation  notwithstanding  the  recognition  that  DOE  is  in  a  transition  state, 
and  the  absence  of  any  analysis  of  why  the  variation  exists.  Neither  does 
the  report  consider  the  variation  in  broader  context.  There  are  a  number  of 
reasons  for  the  differences.  They  include:  1)  significant  differences  in  the 
scope,  magnitude,  and  complexity  of  site  work  activities;  2)  the  timing  cf 
the  contract  structuring  (contracts  awarded  shortly  after  the  adoption  of 
Contract  Reform  recommendations  are  different  than  recently  awarded 
contracts  where  lessons  learned  and  more  seasoned,  sophisticated 
approaches  have  been  applied  or  recent  policies  have  been  adopted;  3)  the 
nature  of  the  proposals  received  in  competitive  awards  and  the  negotiated 
terms  in  contract  extensions;  4)  variations  in  the  status  of  the  contracting 
party  (e.g.  profit  not-profit,  educational  institutions.) 

Page  4.  f  L  The  conclusion  that  contract  differences  are  the  product  of  "DOE’s 

~~  philosophy  of  relying  on  DOE  field  units  to  tailor  contracts  to  local 

conditions  and  contractor  preferences"  is  a  rather  odd  and  unsupported 
statement  DOE  relies  on  its  field  organizations  to  negotiate  and  award 
contracts  associated  with  their  assigned  responsibilities.  So  does  virtually 
every  Federal  organization.  They  are,  however  subjected  to  headquarters 
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review.  Our  respective  contracts  do  reflect  "local  conditions"  such  as  the 
specific,  work  io  be  performed  at  a  site.  Contractor  "preferences"  are 
generally  irrelevant  except  to  the  extent  that  they  are  reflected  in 
competitive  proposals  or  become  contract  negotiation  issues.  This  is  no 
different  than  any  other  Federal  or  commercial  contracting  environment.  In 
any  event,  the  variation  noted  by  GAO  is  more  accurately  reflective  of  the 
factors  previously  noted. 

Page  4. 2,  a)  GAO  notes  that  DOE  has  "struggled"  to  find  the  right  mix  of 

performance  measures.  DOE  is,  admittedly,  still  "growing"  in  its  use  of 
performance  measures  and  objectives.  It  would,  however,  be  helpful  for 
GAO  to  officially  observe  what  its  staff  has  readily  admitted:  the 
development  of  an  "optimum"  set  of  performance  measures  in  any 
environment  is  a  difficult  challenge,  and  perhaps  an  impractical  objective. 
Major  corporations  have  "struggled"  with  performance  measures  since  the 
mid  1980s  and  still  freely  admit  that  they  are  not  satisfied.  Federal 
attempts  at  creating  such  measures  for  its  programs  have  likewise  struggled 
since  the  early  1990s.  GAO  itself  has  not  found  the  subject  matter  easy  and 
has  recognized  the  difficulties  associated  with  any  nascent  concept.  Its 
own  lack  of  expertise  in  this  area  is  reflected  in  the  fact  that  it  has  hired 
consultants  to  advise  it  on  the  subject  and  that  it  has  used  other 
organizations,  including  DOE,  for  benchmarking  purposes. 

b)  The  report  correctly  notes  that  in  its  initial  application  of  performance 
objectives  and  measures  DOE  contracts  had  a  tendency  to  identify  a 
relatively  large  number  of  measures.  Notwithstanding  the  quote  from  an 
unidentified  "field  official",  there  was  in  fact  no  headquarters  direction  to 
create  "large  numbers  of  performance  measures."  There  was,  however,  a 
great  deal  of  management  emphasis  placed  on  moving  to  defined 
performance  objectives  and  criteria.  Overdoing  the  number  of  measures  is 
a  rather  natural  tendency  in  any  organization  initiating  a  move  towards 
performance  management.  It  is  largely  reflective  of  both  a  general  fear  of 
leaving  something  important  out  and  a  desire  to  satisfy  all  stakeholder 
interests.  Interestingly,  it  is  a  tendency  that  is  frequently  exacerbated  by 
oversight  bodies  which  frequently  recommend  that  every  perceived 
deficiency  be  addressed  with  a  specific  performance  standard  and  measure. 
Although  well  meaning,  these  tendencies  are  counterproductive  to  effective 
performance  management.  The  guidance  that  has  been  issued  by  DOE  on 
this  subject  has  pointed  this  out  and  as  a  consequence  there  has  been  a 
general  effort  to  identify  the  "critical  few."  It  is  noteworthy  that  GAO  had 
a  similar  learning  experience  with  this  concept  in  its  early  consideration  of 
performance  measurement  and  has  only  recently  recognized  that  in  this 
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case  more  is  not  necessarily  better. 

Page  S.  3 1.  a)  The  report  notes  that  DOE  is  still  attempting  to  develop  the  “right 

number  of  measures'1  and  notes  that  the  contract  for  the  management  of 
Ames  Laboratory  has  7  while  that  for  the  Idaho  National  Engineering  and 
Environmental  Laboratory  (INEEL)  has  250.  There  is  in  fact  no  "right" 
number  of  measures  for  contracts  for  the  management  of  DOE  laboratories 
and  it  is  unlikely  that  there  ever  will  be.  A  careful  analysis  of  the  contract 
work  scope  of  Ames  verses  the  activities  performed  at  INEEL  might  offer 
some  insight  as  to  why  these  contracts  are  ever  likely  to  have  the  same  or 
the  same  number  of  performance  measures, 

b)  The  report  notes  that  DOE  and  its  contractors  are  working  to  develop 
measures  that  reliably  address  the  most  important  activities  at  the  lab.  This 
is  entirely  accurate.  In  fact,  based  on  the  comments  of  a  private  analyst 
who  assessed  performance  management  in  a  scientific/research  and 
development  environment,  DOE  may  be  the  only  agency  aggressively 
pursuing  the  subject  matter. 

c)  The  report  repeats  a  comment  from  an  unidentified  field  official  that 
"early  attempts"  at  performance  measures  resulted  in  contractors  focusing 
an  what  was  measured  and  incentivized  at  the  risk  of  neglecting  that  which 
was  not.  It  is  unclear  what  significance  GAO  intends  the  reader  to  draw 
from  this  unremarkable  and  fairly  inane  comment.  This  is,  in  fact,  the 
whole  point  of  measurement  and  incentivization.  Is  the  alternative  to  not 
define  expectations,  not  measure  critical  performance  indicators,  and  not 
provide  some  consequence  for  the  result?  Is  the  alternative  to  establish 
contractual  performance  objectives  and  objectives  for  every  conceivable 
work  activity?  If  GAO’s  point  is  to  inform  the  reader  that  effective 
performance  measurement  is  an  evolving  process,  we  wholeheartedly 
agree.  The  comment,  however,  seems  inapt  for  this  purpose.  Equally 
inane  is  the  commenter  s  statement  that  performance-based  contracting 
focuses  on  monetary  reward  not  good  science.  The  statement  is  also 
incorrect.  If  the  commenter  is  observing  that  some  contractors  are 
motivated  by  financial  reward,  then  it  is  rather  obvious  that  linking  such 
rewards  to  performance  may  be  a  useful  idea.  If  the  commenter  is 
observing  that  some  contractors  are  motivated  by  non-monetary  rewards, 
then  it  is  rather  obvious  that  linking  such  rewards  to  performance  may  be  a 
useful  idea.  It  is  for  these  reasons  that  DOE  policy  does  not  require 
laboratory  contractors  to  accept  fee  and  that  DOE  believes  that  there  is 
value  in  defining  performing  expectations  and  measuring  results  even 
where  fee  is  not  available  or  has  otherwise  not  been  associated  with 
targeted  achievements.  Other  motivations,  such  as  reputation  in  the 
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scientific  community,  the  ability  to  earn  extended  contract  performance 
under  options,  and  the  ability  to  receive  new  contract  awards  or  expanded 
assignments  have  been  applied  by  the  Department. 

Page  6.  f  h  a)  The  report  notes  a  snapshot  observation  that  fee  arrangements  vary  among  the 
laboratory  contracts  and  that  9  out  of  19  have  fee  at  performance  risk.  This 
statement  is  not  particularly  accurate  or  informative  of  the  current  status.  DOE 
has  18  contractor  managed  laboratories,  not  19  (ORISE  is  no  longer  designated  a 
DOE  laboratory  and  has  been  dropped  from  the  M&O  system),  16  of  those 
contracts  provide  fee  to  the  contractor  (SLAC  and  Princeton  contain  no  or  a 
nominal  fee  respectively);  of  those  16, 10  contracts  currently  contain  performance 
fee  requirements;  and  1  is  currently  being  competed  with  a  performance  fee.  Of 
the  5  remaining  contracts,  1  was  competitively  awarded  in  1992  as  a  fixed  fee 
contract  and  its  full  term  has  not  expired  (it  will  be  competed  with  fee  linked  to 
performance  upon  completion  of  the  term),  and  2  are  Naval  Reactors  laboratories 
jointly  managed  by  statutory  direction  with  the  Department  of  the  Navy  which 
desires  to  maintain  a  fixed-fee.  As  previously  noted  to  the  auditors  this  "snapshot" 
of  current  status  reflects  the  Department’s  policy  preference  for  performance- 
linked  fee  structures  and  the  migration  to  such  structures  as  appropriate 
opportunities  (contract  termination)  for  change  arise.  Rather  than  concluding  that 
fee  approaches  "vary  widely",  a  more  correct  observation  would  be  that  DOE  is 
systematically  introducing  performance-linked  fees  into  its  laboratory  contracts. 

By  the  end  of  FY  99  the  vast  majority  of  those  contracts  containing  other  than 
nominal  fee  will  link  fee  to  performance. 

Page  6.  £  2,  a)  The  report  states  that  "performance  fees  were  introduced  as  a  way  of 

encouraging  superior  performance."  The  wording  here  is  not  entirely  accurate. 
Performance  fees  were  introduced  in  order  to  link  performance  to  financial  reward, 
as  opposed  to  fixed  fees  which  simply  compensate  work  effort. 

Page  7  &  8.  The  discussion  on  these  pages  highlights  the  vagaries  of  fee  arrangements.  The 
report  does  not  fully  explain  why  variation  exists.  First,  if  should  be  noted  that 
DOE  policy  treats  companies  organized  for  profit  differently  than  nonprofit 
organizations  in  certain  respects.  Among  the  policy,  the  policy  distinctions, 
reflected  in  the  so-called  variation,  is  the  availability  of  some  portion  of  the  fee 
amount  as  a  base  fee.  DOE  generally  will  not  provide  base  fee  for  for  profits,  but 
will  consider  doing  so  for  nonprofit  organizations  in  order  to  provide  financial  risk 
pools  not  otherwise  available.  Secondly,  the  work  activity  at  DOE  laboratory  sites 
is  in  many  cases  dramatically  different.  Specifically,  one  site  may  be  engaged 
almost  entirely  in  basic  scientific  research;  another  may  be  focused  on  applied 
research  with  emphasis  on  introducing  the  results  into  the  industrial  market  place; 
a  third  may  be  engaged  in  extensive  effort  to  remediate  legacy  environmental  and 
health  hazards;  a  fourth  may  be  concentrating  on  the  technical  aspects  of  weapons 
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systems  management;  a  fifth  may  have  an  extensive  facility  construction  program; 
a  sixth  may  have  all  the  above.  In  short,  DOE  laboratories  do  not  come  out  of  a 
cookie  cutter  and  the  appropriateness  and  utility  of  award  fee  and  incentive  fee 
arrangements  may  differ  significantly.  Finally,  there  is  the  impact  of  competitive 
proposals  and  the  contract  negotiation  process.  To  put  GAO’s  observate  in 
broader  context  it  is  noteworthy  that  variation  in  fee  approaches  is  not  an 
abnormality  in  government  contracts  generally,  and  certainly  not  in  laboratory  and 
research  and  development  contracts.  Our  review  of  other  agencies  contracts  for 
these  purposes  indicates  a  broad  array  of  approaches  throughout  the  government 
and  within  individual  agencies.  It  would  appear  that  no  single  approaches  has 
proven  to  be  the  optimum  template  for  such  contracts. 

Paye  9.  IT  1.  The  report  contains  a  statement  that  it  is  DOE  policy  that  the  purpose  of 

performance-based  contracting  is  to  obtain  better  performance  and  if  these  results 
are  not  achieved  it  isn’t  worth  doing.  Actually,  the  statement  is  not  DOE  policy, 
but  rather  is  taken  from  a  forward  in  a  guide.  The  forward  was  written  by 
someone  outside  the  Department.  Further  we  note  that  GAO  failed  to  quote  the 
entirety  of  his  statement  which  indicated  that  there  was  no  procurement  where  it 
cannot  be  successful.  The  report  then  notes  that  DOE  has  not  analyzed  the  impact 
of  performance-based  contracts  on  its  laboratories,  with  the  implication  that  it  may 
not  be  worth  doing.  DOE  has  in  fact  performed  a  number  of  assessments  of  the 
concept,  even  at  this  early  stage  of  application,  and  has  concluded  that  it  is  having 
a  positive  impact  Although,  it  is  unlikely  that  in  the  future  we  will  identity  some 
scientifically  conclusive  mechanism  to  confirm  or  deny  these  early  assessments,  it 
would  appear  intuitively  obvious  that  defining  performance  expectations  and 
measuring  results  is  an  effective  management  tool.  It  could  also  appear  intuitively 
obvious  that  measurement  without  consequence  or  reaction  is  an  imperfect 
approach.  We  assume,  for  example,  that  after  years  of  GPRA  implementation 
without  the  conduct  of  any  merit  assessment,  GAO  still  believes  that  the  concept  is 
valid  and  worthy  of  continued  implementation.  The  GAO  discussion  in  this  regard 
appears  to  focus  on  the  question  of  whether  higher  contract  fees  are  a  worthwhile 
consequence  of  performance  focus.  In  part,  this  discussion  is  confused  by  other 
DOE  actions  with  respect  to  1)  the  elimination  of  the  concept  of  a  management 
allowance  for  nonprofit  laboratory  contractors,  2)  the  upgrading  of  DOE  fee 
schedules  to  take  into  account  years  of  inflation,  and  3)  the  introduction  of  greater 
financial  risk  into  management  and  operating  contracts  generally  and  with  respect 
to  laboratory  contracts  in  particular.  In  any  event,  government-wide  profit  policies 
generally  recognize  that  potential  fee  at  risk  instruments  is  greater  than  that 
associated  with  fixed  fee  instruments.  This  is  also  rather  intuitive. 

Page  10, a)  The  report  states  that  without  an  overall  cost  benefit  analysis  it  is  difficult  to 

determine  the  value  of  $100  million  spent  on  contractor  fees  for  1998.  We  are  not 
aware  of  any  model  that  has  been  developed  or  used  by  any  agency  for  such  an 
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assessment,  and  as  far  as  we  could  determine  from.  GAO  auditors,  neither  has 
GAO.  Neither  has  GAO  offered  any  meaningful  advice  on  howto  construct  such 
a  model.  Further,  there  appears  to  be  an  implication  that  absent  such  analysis  fee 
should  not  be  paid  for  our  contracts.  Such  a  conclusion  would  quickly  resolve  any 
issues,  because  it  is  unlikely  that  future  contracts  would  ever  be  awarded. 

It  is  also  worthwhile  to  put  the  total  fee  amount  for  the  subject  contracts  in 
perspective:  1)  A  number  of  the  contracts  reviewed  by  GAO  are  for  a  scope  of 
operations  that  go  far  beyond  typical  laboratory'  contracts  (e.g.  INEEL  and  the 
Savannah  River  Site.)  These  contracts  account  for  a  significant  portion  of  total  fee 
noted  by  GAO;  2)  The  typical  fee  paid  to  laboratory  contractors  is  among  the 
lowest  paid  under  any  government  contract,  ranging  from  0%  to  .02%  to  .06%  to 
L8%  of  the  contract  cost  base;  3)  A  number  of  the  arrangements  limit  the  use  of 
fee  earned  to  laboratory  related  scientific  research  and  financial  risk  pools:  and 
4)  The  applications  of  performance-based  concepts  has  resulted  in  greatly  reduced 
fee  earnings  in  contracts  where  performance  has  not  met  expectations.  As  a 
consequence,  fee  pay  cuts  have  been  reduced  for  sub-optimal  performance  by  up 
to  100%. 

general 

a)  The  report  could  be  improved  by  greater  reference  to  the  concepts  articulated  in 
DOE’s  regulation  on  fee. 

b)  There  are  a  number  of  data  inaccuracies  in  the  report  attachments.  We  will  be 
happy  to  provide  corrected  information  at  GAO'S  convenience. 
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GAO  Comments 


The  following  are  gao’s  comments  on  the  Department  of  Energy’s  letter 
dated  April  22, 1999. 

1.  We  have  made  changes  to  the  report  as  appropriate  in  response  to  doe’s 
comments. 

2.  Our  wording  is  drawn  from  doe’s  guidance  on  performance-based 
contracting,  and  we  have  made  changes  to  our  report  to  reflect  doe’s 
comments,  doe  recommends  that  its  laboratory  contracts  contain 
performance-based  features,  which  include  clear  expectations  described 
in  terms  of  results,  not  how  the  work  is  to  be  accomplished. 

3.  As  we  stated  in  our  report,  doe’s  evaluations  did  not  focus  on  the 
laboratory  contractors,  nor  did  these  evaluations  focus  on  the  costs  and 
benefits  of  performance-based  contracting  features,  including  the  impact 
of  fees. 

4.  We  recognize  that  one  of  the  purposes  of  providing  fees  is  to  reflect  the 
financial  risk  associated  with  work  performance,  and  we  make  this  point 
in  our  report.  Our  1994  recommendation  questioned  the  cost-benefit  of  the 
increased  fees,  regardless  of  whether  they  were  related  to  performance  or 
financial  risk.  We  continue  to  believe  that  our  recommendation  is  relevant 
because  doe  has  not  evaluated  the  cost  and  benefit  of  the  fees  it  is 
providing  to  laboratory  contractors. 

5.  We  believe  our  wording  adequately  reflects  the  conditions  discussed. 
Information  on  the  laboratory  fees  and  total  contract  costs  is  presented  in 
appendix  I. 

6.  We  have  made  changes  to  the  report  as  appropriate  in  response  to  doe’s 
comments  on  contract  type.  We  stated  in  our  report  that  doe’s 
performance-based  contracting  is  in  a  state  of  transition.  We  also  stated 
that  there  are  wide  variations  in  performance  measures  and  fee 
arrangements  negotiated  by  doe  and  its  laboratory  contractors.  This 
material  is  presented  as  facts  describing  the  conditions  that  presently 
exist.  Our  report  also  describes  the  reasons  for  the  variability  in  laboratory 
contracts  and  includes  most  of  the  reasons  given  in  doe’s  comments.  We 
have  made  changes  in  the  report  to  reflect  these  additional  reasons  for  the 
variability  in  doe’s  laboratory  contracts. 

7.  Our  statement  that  contract  differences  are  the  product  of  doe’s  relying 
on  its  field  units  to  tailor  contracts  to  local  conditions  is  based  on 
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interviews  with  numerous  doe  field  officials.  This  statement  is  not  an 
implied  criticism  of  how  doe  negotiates  contracts.  Also,  we  disagree  with 
doe’s  characterization  that  contractors’  preferences  are  “generally 
irrelevant”  when  accounting  for  the  variations  that  exist  among  laboratory 
contractors.  As  doe  noted,  contractors’  preferences  are  reflected  in  the 
negotiation  process.  In  our  discussions  with  doe  field  officials  responsible 
for  negotiating  contracts,  laboratory  contractors’  preferences  on  fees  were 
cited  as  a  critical  factor  in  determining  fee  structures. 

8.  Our  report  recognizes  that  developing  the  optimum  number  of 
performance  measures  is  a  challenge,  as  reflected  in  the  wide  range  of 
performance  measures  in  use  even  among  similar  laboratories.  We  are  not 
suggesting  that  any  two  contractors  should  have  the  same  measures  or  the 
same  number  of  measures.  Our  point  is  that  doe  continues  to  struggle  with 
finding  the  right  number  of  measures.  To  further  illustrate,  the  University 
of  California’s  fiscal  year  1998  contracts  for  its  two  weapons 
laboratories — Lawrence  Livermore  and  Los  Alamos — contain  83  and  120 
performance  measures,  respectively,  even  though  these  laboratories  are 
very  similar  in  budget  and  scope.  They  are,  however,  managed  by  different 
doe  field  units. 

9.  Our  purpose  in  including  comments  we  received  from  doe  field  units  is 
to  illustrate  the  wide  differences  in  philosophy  about  the  use  of  fees  to 
motivate  laboratory  contractors.  Several  doe  field  staff,  as  well  as 
contractors,  told  us  that  they  strongly  believe  that  providing  fees  does  not 
motivate  contractors,  including  both  for-profit  and  not-for-profit 
contractors.  Moreover,  our  statement  that  performance-based  contracting 
has  tended  to  focus  in  some  instances  on  monetary  rewards  at  the  expense 
of  good  science  was  a  frequent  comment  from  both  doe  field  officials  and 
laboratory  contractors.  Thus,  it  is  very  important  to  identify  the  need  for 
monetary  incentives  where  they  are  appropriate.  Other  motivations  that 
doe  cited  for  laboratory  contractors,  such  as  their  reputations  in  the 
scientific  community  and  desire  for  contract  extensions,  were  added  to 
our  report.  These  differences  in  philosophy  account  for  some  of  the 
variation  in  contracts. 

10.  Our  report  reflects  information  provided  directly  from  doe  field  staff, 
who  we  were  advised  by  doe  headquarters  were  the  proper  source  for  this 
information.  The  data  in  doe’s  comments  are  reflected  in  the  appendixes 
to  our  report.  We  have  also  revised  our  report  to  show  that  there  are  now 
18  laboratory  contractors,  reflecting  a  recent  change  in  how  doe  defines  its 
laboratories. 
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Appendix  III 

Comments  From  the  Department  of  Energy 


11.  dob  field  officials  told  us  that  performance  fees  are  used  to  encourage 
superior  performance.  Asserting  that  fees  are  used  to  link  performance  to 
financial  reward  is  self-evident  in  this  context. 

12.  We  agree  with  doe  that  no  single  approach  in  contracting  has  proven  to 
be  optimum,  and  we  reflected  this  view  in  our  report.  Regarding  the  wide 
variability  in  fee  arrangements,  we  stated  that  there  was  very  little 
consistency  among  the  contracts  of  similar  laboratory  contractors 
conducting  similar  work.  We  also  stated  that  local  conditions  influence  the 
variability  in  laboratory  contracts. 

13.  Our  wording  was  taken  from  doe’s  guidance  on  performance-based 
contracting.  As  we  state  in  our  report,  prior  assessments  of 
performance-based  contracting  have  not  focused  on  laboratory 
contractors.  We  also  stated  in  our  report  that  doe  believes  that  the  results 
from  its  assessments  of  performance-based  contracting  have  been 
positive.  We  believe  it  is  a  logical  and  desirable  step  for  doe  to  determine 
whether  performance-based  contracting  is  improving  performance  at 
lower  cost  in  its  national  laboratories.  Also,  we  are  not  suggesting  that  doe 
should  abandon  its  performance-based  approach  or  that  it  should 
eliminate  performance-based  fees  in  its  laboratory  contracts.  It  is  also  not 
our  intent  to  show  that  performance-based  contracting  should  be 
abandoned  if  its  impacts  on  the  laboratories  cannot  be  measured.  We 
believe  that  effective  implementation  of  performance-based  contracting 
provisions  is  dependent  on  the  ability  to  support  the  fee  amounts  paid 
through  a  cost  and  benefit  analysis.  While  it  may  appear  intuitively 
obvious  that  defining  performance  expectations  and  measuring  results  are 
effective  management  tools,  it  is  not  intuitively  obvious  that  the 
government  is  receiving  a  reasonable  return  on  its  investments  in  fee 
amounts  for  laboratory  contractors.  Likewise,  while  doe  commented  that 
increases  in  fees  reflect,  in  part,  the  increased  financial  risks  being  borne 
by  contractors,  no  cost-benefit  analysis  quantifying  this  increased  financial 
risk  has  been  completed;  thus  it  is  not  possible  to  determine  if  the  proper 
level  of  fee  is  appropriate  for  the  risk  assumed. 

14.  We  recognize  that  laboratory  contractor  fees  are  relatively  small 
percentages  of  the  total  contract  amounts.  However,  these  percentages, 
which  translated  into  $100  million  in  fees  for  fiscal  year  1998,  must  be 
considered  in  light  of  the  fact  that  doe’s  laboratories  are  government 
owned  and  that  a  laboratory  contractor’s  financial  risk  is  limited. 
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Appendix  IV 


Scope  and  Methodology 


To  obtain  information  on  the  national  laboratories’  contracts,  we 
interviewed  officials  from  the  following  laboratories:  Sandia  National 
Laboratories  and  Los  Alamos  National  Laboratory  in  New  Mexico; 
Lawrence  Berkeley  National  Laboratory,  Lawrence  Livermore  National 
Laboratory,  and  the  Stanford  Linear  Accelerator  in  California;  the  National 
Renewable  Energy  Laboratory  in  Colorado;  the  Idaho  National 
Engineering  and  Environmental  Laboratory  in  Idaho;  the  Oak  Ridge 
National  Laboratory  in  Tennessee;  and  the  Argonne  National  Laboratory  in 
Illinois.  We  also  spoke  with  laboratory  officials  in  other  locations  to  obtain 
cost  and  status  information.  We  asked  officials  at  these  laboratories  to 
comment  on  the  impact  of  performance-based  contracting  on  their 
operations. 

We  also  interviewed  Department  of  Energy  (doe)  officials  responsible  for 
overseeing  these  laboratories.  These  officials  were  from  doe’s  operations 
offices  in  Albuquerque,  New  Mexico;  Oakland,  California;  Oak  Ridge, 
Tennessee;  and  Chicago,  Illinois.  We  also  interviewed  doe  area  and  site 
office  staff  located  at  each  of  the  operations  offices  we  visited.  To  obtain  a 
broader  perspective,  we  interviewed  doe  headquarters  officials 
responsible  for  developing  contracting  policy. 

We  conducted  our  review  from  September  1998  through  April  1999  in 
accordance  with  generally  accepted  government  auditing  standards. 
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